Sparse Higher Order Functions of Discrete Variables — Representation and Optimization

Higher order energy functions have the ability to encode high level structural dependencies between pixels, which have been shown to be extremely powerful for image labeling problems. Their use, however, is severely hampered in practice by the intractable complexity of representing and minimizing such functions. We observed that higher order functions encountered in computer vision are very often “sparse”, i.e. many labelings of a higher order clique are equally unlikely and hence have the same high cost. In this paper, we address the problem of minimizing such sparse higher order energy functions. Our method works by transforming the problem into an equivalent quadratic function minimization problem. The resulting quadratic function can be minimized using popular message passing or graph cut based algorithms for MAP inference. Although this is primarily a theoretical paper, we also show how labeling problems such as texture denoising and inpainting can be formulated using sparse higher order energy functions. We demonstrate experimentally that for some challenging tasks our formulation is able to outperform various state-of-the art techniques, especially the well-known patch-based approach of Freeman et al. [11]. Given the broad use of patch-based models in computer vision, we believe that our contributions will be applicable in many problem domains.